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(54) Title: SEQUENCE ADAPTIVE BIT ALLOCATION FOR PICTURES ENCODING 

(57) Abstract 

A method and apparatus for encoding 
pictures of a moving pictures sequence ac- 
cording to an overall target bit-rate, such as 
in a MPEG video encoder. Each picture has 
an assigned picture coding type for which 
a quality factor is adaptively determined ac- 
cording to past bit usages, so that bits can be 
adaptively allocated amongst picture types 
for optimizing visual quality of the encoded 
pictures. 
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SEQUENCE ADAPTIVE BIT ALLOCATION FOR PICTURES ENCODING 

Technical Field 

5 The present invention relates to a method and apparatus for encoding moving pictures 
sequences. In particular, the present invention relates to a method and apparatus for bit 
allocation in a video signal compression system. 

Background Art 

10 

Methods for encoding moving pictures or video had been developed for efficient transmission 
and storage. A current art of such encoding methods is found in MPEG2 Test Model 5, 
ISO/IEC JTC1/SC29/WG11/NO400, April 1993, and the disclosure of that document is 
hereby expressly incorporated herein by reference. In this method, an input video sequence is 
15 organized into sequence layer, group-of-pictures, pictures, slices, macroblocks, and finally 
block layer. Each picture in a group-of-pictures will be coded according to its determined 
picture coding type. 

The picture coding types used in the MPEG2 Test Model 5 include intra-coded picture (I- 
20 picture), predictive-coded picture (P-picture), and bi-directionally predictive-coded picture 
(B-picture). The I-pictures are used mainly for random access or scene update. The P- 
pictures use forward motion predictive coding with reference to previously coded I- or P- 
pictures (anchor pictures), and the B-pictures use both forward and backward motion 
predicuve/interpolative coding with reference to previously coded I- or P-pictures. A group 
25 of pictures (GOP) is formed in encoded order starting with an I-picture and ending with the 
picture before the next I-picture in the sequence. 

A picture is partitioned into smaller and non-overlapping blocks of pixel data called 
macroblocks (MB) before encoding. Each MB from a P- or B-picture is subjected to a 
30 motion estimation process in which forward motion vectors, and backward motion vectors 
for the case of a B-picture MB, are determined using reference pictures from a frame 
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buffer. With the determined motion vectors, motion compensation is performed where the 
intra- or inter-picture prediction mode of the MB is first determined according to the 
accuracy of the motion vectors found, followed by generating the necessary predicted MB 
containing the prediction error. 

5 

The predicted MB is then subjected to discrete cosine transform (DCT) and quantization of 
the DCT coefficients based on quantization matrices and quantization step-size. The 
quantized DCT coefficients of the MB is then run-length encoded with variable length codes 
(VLC) and multiplexed with additional information such as selected motion vectors, MB 
10 coding modes, quantization step-size, and/or picture and sequence information, to form the 
output bitstream. 

Local decoding is performed by inverse quantizing the quantized DCT coefficients, followed 
by inverse DCT, and motion compensation. Local decoding is performed such that the 
1 5 reference pictures used in the motion compensation are identical to those used by any 
external decoder. 

The quantization step-size (QS) used for quantizing the DCT coefficients of each MB has a 
direct impact on the number of bits produced at the output of the run-length VLC encoding 

20 process, and therefore the average output bit rate. It has also a direct impact on the encoding 
quality, which represents the output picture quality at the corresponding decoder. In general, 
larger QS generates lower output bit rate and lower encoding quality. In order to control 
output bit rate and picture quality so that the resulting bitstream can satisfy channel 
bandwidth or storage limitation as well as quality requirements, rate control and quantization 

25 control algorithms are used. 

Some methods for rate control and quantization control can be found in the above mentioned 
MPEG-2 Test Model 5. These methods comprise generally a bit allocation process, a rate 
control process, and an adaptive quantization process. In the bit allocation process, a target 
30 number of bits is assigned for a new picture to be coded according to a number of previously 
determined and pre-set parameters. The rate control step then calculates a reference 
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quantization step-size for each MB based on the target bits for the picture and the number of 
bits already used from the target bits in encoding MBs from that picture. In the adaptive 
quantization process, the calculated reference quantization step-size is scaled according to 
local activities of the MB, and an average MB activity determined from the current or a 
5 previously coded picture. This scaling is done according to a level of masking effects of 
coding noise by human perception for MB with high or low activities within a picture. A 
video buffer verifier (VBV) may also be employed in such a way that underflow and overflow 
of the decoder input buffer are prevented as required by the MPEG standard to ensure a 
target bit rate is maintained. 

10 

It is assumed in the bit allocation process that the visual quality of a coded picture can be 
qualified with a single number V e , expressed by the formula: 




where Q is the average quantization step-size of the coded picture and K is a constant 
quality factor which depends only on the picture coding type. It is also assumed that the 
15 visual qualities of all encoded pictures should be maintained at a similar level within a 
GOP. Therefore, for all pictures within a GOP, the bit allocation process maintains the 
following equality: 

K, K p K B 

Qi" Qp" Q B (2) 

where Q T , Q r , Q B are the respective average quantization step-sizes of coded I-, P- ,and B- 
picture, and similarly K r , Kp, K B are respective pre-determined quality factors for I-, P- 
20 ,and B-pictures. Although this equality does not apply to an entire pictures sequence, it 
should be considered valid within a GOP as well as across consecutive GOP boundaries. 
For simplicity, K„ of equation (2) is normalized to the value of 1. 



From the above assumptions, an equation for determining target bit allocation for a picture 
25 to be coded can be derived for each of the picture coding types. The equations are given 
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as follows for each of the I-, P- and B-picture coding type: 




5 where 

X, = S, Q t , X P = S P Qr, X B = S B Q B , 

and 

S Jt Sp, S B are number of bits used by previously encoded I-, P-, B-picture 
10 respectively, 

Qi. Qp. Q B are the average quantization step-size used by previously encoded I-, P- 
, B-picture respectively, 

N P and N„ are the number of P- and B-pictures remaining in the current GOP with 
respect to the current picture to be coded, 
15 R is the remaining number of bits assigned to the GOP according to a target bit- 

rate, and 

Tj, Tp, T B are the calculated target bit allocation for a new I-, P-, B-picture to be 
coded respectively. 
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An optional lower limit may be applied to the determined target bit allocation as given in 
the MPEG2 Test Model 5, hence: 

(1. or I p or Tr.) z = /A x 

1 P B/ K, * Picture_Rate < 6 > 

where Bit Rate is the target bitrate, 

Picture Rate is number of pictures coded per second, and 
5 Kj is a constrant (eg. 8). 

A typical video encoder system is designed to code picture sequences with various 
characteristics and complexities. In particular, sequences with little motion updates or 
complex motion scenes creates different requirements for coding pictures of difference 
10 picture coding types. 

For example, a sequence with little motion updates may be best coded with higher ratio of 
bits allocated to the anchor pictures (I-pictures and P-pictures) for visual quality 
improvements. On the other hand, a sequence with complex motion scenes may be best 
15 coded with relatively even distribution of bits to pictures of all picture coding types for 
motion detail improvements, and hence higher ratio of bit allocation to the B-pictures. 
Present systems based on fixed visual quality ratios, for example according to equation 
(2), do not adequately address these changes in sequence characteristics. 

20 In other words, the pre-determined and fixed quality factors K„ K P and K B have more or 
less determined the ratios of the average quantization step-sizes (ie. Q T , Q P , and Q B ratios) 
to be used for coding pictures of different picture coding types. This relationship limits 
the adaptivity of bit allocation to different picture coding types of different motion 
characteristics. 

25 

Furthermore, the adaptivity of bit allocation should depend on the visual quality itself. 
When higher visual quality is achieved for the anchor pictures, a feature which re- 
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distributes the bits to B-pictures is desired such that both visual quality and motion details 
can be balanced. 

Summary of the Invention 

5 

It is an object of the present invention to provide a method or apparatus for picture 
sequence compression which better caters for changes in sequence and coding 
characteristics. In particular, an object of the present invention is to provide a method or 
apparatus for improving bit allocation. 

10 

While sophisticated methods of detennining motion characteristics by frame difference 
detection or motion vector analysis may be employed, such methods may have significant 
impact on implementation cost. Implementation cost may be an important consideration 
for systems to be broadly used. A further object of the present invention is therefore to 
15 provide a method or apparatus for bit allocation with low implementation complexity. 

One way of solving the above mentioned problems is to adaptively determine new quality 
factors K„ Kp, and K B of equations (2), (3), (4), and (5) according to characteristics or 
parameters obtained from past coded pictures from a sequence. This is in effect changing 
20 the target ratios of the average quantization step-sizes for I-, P-, and B-pictures such that 
visual quality and/or motion details can be improved depending on the contents within the 
sequence. 

A further simplified approach is to adaptively modify only the value of K B , therefore the 
25 target ratios of the average quantization step-size of B-pictures over that of I- or P- 
pictures', as experimentally determined that the impact is not significant in terms of 
solving above mentioned problems when K, or K P are also modified adaptively. 

The quality factors are modified adaptively in the present invention according to the 
30 average quantization step-sizes and bit usage of past coded I-pictures, P-pictures and B- 
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pictures. Encoding quality and coding complexity of a picture may be measured from its 
average quantization step-size and bits used. The ratios of coding complexities of pictures 
of different picture coding types together with encoding qualities may be used to indicate 
spatial and/or motion complexity, and therefore, the quality factors may be determined 
5 accordingly. 

A video encoding system encodes a video sequence of various picture coding types (I-, P- 
and B-pictures) with sequence adaptive bit allocation according to the present invention. 
Initial values of necessary parameters for coding may be predetermined. Bits are allocated 
10 to each picture to be coded such that the resulting visual qualities defined as K/Q of all 
pictures within a group-of-pictures remains at a similar level. 

The quality factor K is dependent on the picture coding type; and furthermore, K, is 
adaptively modified according to the average quantization step-sizes (Q) and bit usages of 
15 past coded pictures. 

With bits allocated to a picture to be coded, a rate controller is then used to regulate bits 
for coding of each macroblock within the picture. A video encoding system according to 
the present invention may include components that are similar to those used in the MPEG2 
20 Test Model 5. 

Brief Description of the Drawings 

The invention is described in greater detail hereinafter, by way of example only, with 
25 reference to a preferred embodiment thereof and the accompanying drawings, wherein: 

Fig. 1 is a block diagram illustrating a video encoding system with sequence 
adaptive bit allocation according to an embodiment of the present invention; and 

Fig. 2 is a flow chart of an embodiment of a Quality Factor Estimator of present 
invention. 



30 
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Detailed Description of the Preferred Embodiment 

Figure 1 illustrates a video encoding system 100 with sequence adaptive bit allocation 
according to an embodiment of the present invention. An input video picture sequence is 
5 encoded by a video encoder 101 to produce a compressed output bitstream which may be 
transmitted to an external decoder via a communication channel or recorded on digital 
storage media for playback applications. Each picture from the input video sequence is 
coded according to its assigned picture coding type (I-, P-, or B-picture). A group-of- 
pictures (GOP) is formed starting with an I-picture, and followed by possibly sets of P- 
10 pictures and B-pictures. 

Typically, the video encoder 101 utilizes motion estimation, motion compensation, 
discrete cosine transform (DCT) coding, and mn-length/differential encoding with variable 
length codes (VLC) as video compression techniques. A video encoder may be 
15 implemented such that the output bitstream is compliant to one of the ISO/IEC MPEG 
standards, and can be decoded by the corresponding MPEG video decoders. 

A bit allocation processor 103 is coupled to or forms part of the video encoder, and is 
provided to determine a target number of bits (T) to be used for coding of a picture from 

20 the input video according to its picture coding type and target bit-rate. The target bit-rate 
and picture coding type are provided as input to the bit allocation processor 103. Bits are 
allocated so that the resulting visual qualities defined as K/Q of all pictures within a 
group-of-pictures remains at a similar level. The quality factor K depends on the picture 
coding type (eg. K t , K P , K„) and Q is the average quantization step-size used for coding 

25 that picture type. Example methods of bit allocation may be found in MPEG2 Test Model 
5, as given by equations (3), (4), (5) with an optional lower limit given by (6). A set of 
initial values may be assumed for K„ Kp, Kg, X r , X P and X B . 

After determining the target bits (T), a bit rate controller 104 and an adaptive quantization 
30 processor 105 are used to calculate the quantization step-sizes for coding of each 
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macroblock (MB) in the picture. The quantization step-sizes are used optionally together 
with quantization matrices to quantize the MBs within the video encoder 101. The MBs 
may be subjected to motion estimation, motion compensation, and discrete cosine 
transform prior to quantization. Frame field adaptive methods mentioned in MPEG2 Test 
5 Model 5 may also be applied. The quantized MBs are run-length encoded with variable 
length codes and multiplexed with all necessary side information to provide the 
compressed output bitstream. 

The rate controller 104 calculates a reference quantization step-size for each MB based on 
10 the target bits (T) for the pictures and the number of bits consumed from the target bits in 
encoding MBs from that picture. For adaptive quantization 105, the calculated reference 
step-size is scaled according to local activities of the MB and an average MB activity 
determined from the current or a previously coded picture. Methods of rate control and 
adaptive quantization may also be found in MPEG2 Test Model 5. 

15 

A quality factor estimator 102 is used to estimate the values of K according to the picture 
coding type and bit usages (S) of past coded pictures supplied as input from the video 
encoder, and the average quantization step-sizes (Q) which are supplied from the adaptive 
quantization processor 105. Together with the average quantization step-size Q, the 

20 quality factor K determines the visual quality of an encoded picture. A set of values for K 
may be determined for each picture coding type. Hence, K„ Kp, and K B may be 
adaptiveiy determined. Furthermore, the set of values for K may be updated at for each 
picture or for every plurality of pictures, according to computation and visual quality 
adaptivity needs. The estimated values of K are used by bit allocation processor 103 to 

25 determine the target number of bits to be used for coding of each picture. 

Figure 2 illustrates a flow chart 200 of operational steps of the quality factor estimator 
according to an embodiment of the present invention. Starting with step 201 , coding 
information such as the picture coding types, the average quantization step-sizes and bit 
30 usages of coded pictures are obtained. For step 202, encoding qualities are determined 
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using the average quantization step-sizes. The encoding qualities for past coded pictures 
of different picture coding types are determined for the purpose of re-distributing bits to 
B-pictures when good encoding quality for P-pictures is achieved. Similarly, bits are re- 
distributed to I-pictures when good encoding qualities for P-pictures and B-pictures are 
5 achieved. For example, it is possible to set a maximum and minimum limit for the value 
of K of a particular picture coding type using average quantization step-sizes for past 
coded pictures as one such method. In particular, a simplified embodiment may be given 
as follows: 

K b-max = C / * Qp + C 2 , 

10 where C 0 to C 4 are constants (eg. 1.4, 0.3, -0.4, 1.4, 2.0 respectively), 

Q P is the average reference quantization step-size of a past coded P-picture, and 
k b-max and Kihmn are the maximum and minimum quality factor K for B-pictures. 

Similarly, K,.^, K r . MlN , Kp.^, and K P , Mm for I-pictures and P-pictures may be 
15 determined. In fact, for simplicity K umx , K hmf , K^, and K P . MIN may also be 
normalized to the constant value of 1 . 0. 

At 203, the coding complexities (X„ X P , X B ) of past coded pictures are calculated. The 
coding complexity indicates spatial and/or motion complexity in coding picture of a 
20 particular picture coding type. It is given as: 

' Sj * Qj 
X p = S p x Q p 

where S„ Sp, S B are the number of bits generated by encoding a past I-picture, P-picture 
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and B-picture respectively, and 

Qi, Qp, Qb are the average quantization step-size used for encoding all macroblocks 
of the I-picture, P-picture and B-picture respectively. 

5 With the determined encoding qualities and coding complexities, the quality factors (K r , 
Kp, and K B ) are estimated at the next step 204. The ratios of coding complexity of 
different picture coding type are used to define the ratios of the quality factors such that 
the bit allocation process can take into consideration spatial and/or motion complexities of 
different picture coding types. Furthermore, the ratios of the quality factors are also 
10 limited according to the determined encoding qualities so as to maximize visual quality 
and motion details. For simplicity, the quality factors can be normalized according to the 
following example: 

X 

Kl = Cs * ~Xj + C(i ' md set * K * * K '- MAX ( 7 ) 

K p = 1.0, (8) 
X 

K » = C t*T + C *' md SCt Kb -*™ * K * * K »-"*x ( 9 ) 



where C s to C„ are constants. 

15 The values of C, to C„ may be experimentally determined. It is also possible to further 
simplify the method by setting C, to C, to values of 0, 1, 1, 0 respectively. 

The bit allocation process attempts to maintain similar visual quality which is based on the 
average quantization step-sizes and the quality factors (K„ Kp, and K B ) while encoding 
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pictures or groups of pictures from an input video sequence. By adaptively modifying the 
quality factors according to picture coding types, and encoding qualities and coding 
complexities determined by the average quantization step-sizes and bit usages of coded 
pictures, a video encoder according to an embodiment of the present invention can 
5 efficiently allocate bits from a target bit-rate to encode each pictures within the video 
sequence. 

The quality factors are adapted in such a way that more bits are used for motion detail 
improvements when there are complex motion scenes within the picture sequence; and on 
10 the other hand, more bits are allocated to improve visual (spatial) quality when scenes 
within picture sequence contain little motion updates. The bit allocation is further checked 
with the encoding qualities achieved so that the both motion details and visual quality can 
be optimized to enhance the overall objective results of the video encoder. 



15 
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Claims 

1. A moving pictures encoder for encoding pictures in a moving pictures sequence 
according to an overall target bit-rate, each picture having an assigned picture coding type, 

5 the moving pictures encoder including: 

a quality factor estimator for determining a quality factor according to the assigned 
picture coding type, bits used for past coded pictures of that type, and an average 
quantization step size; 

a bit allocation processor for determining a target number of bits for coding a picture 
10 according to the overall target bit-rate, the assigned picture coding type and the corresponding 
quality factor; 

a bit-rate controller for determining a reference quantization step-size based on the 
target number of bits and the number of bits used for the picture; and 

an adaptive quantization processor for determining a coding quantization step-size 
15 based on the reference quantization step-size and picture activities for the picture, and for 
determining said average quantization step-size for past coded pictures. 

2. A moving pictures encoder as claimed in claim 1, wherein a corresponding quality 
20 factor is determined by the quality factor estimator for each picture type. 

3. A moving pictures encoder as claimed in claim 2, wherein a value for the quality 
factor is determined for coding of each picture of the corresponding picture type. 

25 4. A moving pictures encoder as claimed in claim 2, wherein a value for the quality 
factor is determined for coding of each plurality of pictures of the corresponding picture type. 



30 



5. A moving pictures encoder as claimed in claim 2, 3 or 4, wherein minimum and 
maximum limits are set for the quality factor of each picture type. 
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6. A method for encoding pictures in a moving pictures sequence according to an overall 
target bit-rate, each picture having an assigned picture coding type, including the steps of: 

determining a quality factor according to the assigned picture coding type, bits used 
for past coded pictures of that type, and an average quantization step size; 
5 determining a target number of bits for coding a picture according to the overall target 

bit-rate, the assigned picture coding type and the corresponding quality factor; 

determining a reference quantization step-size based on the target number of bits and 
the number of bits used for the picture; 

determining a coding quantization step-size based on the reference quantization step- 
10 size and picture activities for the picture, and for determining said average quantization step- 
size for past coded pictures; and 

encoding a picture according to the determined coding quantization step size. 

7. A method as claimed in claim 1, wherein a corresponding quality factor is determined 
15 for each picture type. 

8. A method as claimed in claim 2, wherein a value for the quality factor is determined 
for coding of each picture of the corresponding picture type. 

20 9. A method as claimed in claim 2, wherein a value for the quality factor is determined 
for coding of each plurality of pictures of the corresponding picture type. 

10. A method as claimed in claim 2, 3 or 4, wherein minimum and maximum limits are 
set for the quality factor of each picture type. 

25 
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